Crowdsourced Security Vulnerability Discovery: Modeling and Organizing Bug-Bounty Programs

Despite significant progress in software-engineering practices, software utilized for desktop and mobile computing remains insecure. At the same time, the consumer and business information handled by these programs is growing in its richness and monetization potential, which triggers significant privacy and security concerns. In response to these challenges, companies are increasingly harvesting the potential of external (ethical) security researchers through bug bounty programs to crowdsource efforts to find and ameliorate security vulnerabilities [5,10]. These so-called white hat hackers are often rewarded with monetary bounties and public recognition. Broadening the appeal of crowdsourced security, several commercial bug bounty platforms have emerged (e.g., HackerOne, BugCrowd, Cobalt) and successfully facilitate the process of building and maintaining bug bounty programs for organizations. For example, on HackerOne, more than 20,000 security vulnerabilities have been reported and fixed for hundreds of organizations. Contributions came from over 2500 different white hat hackers who received bounties of over $7.3M as of May 2016. Over the last two years, we have begun to systematically study these platforms from an empirical perspective to evidence their growing popularity and practical contributions to the security of deployed code [10,9]. While empirical results imply that bug bounty programs make a significant contribution to security, there also exists several obstacles for running and scaling bug bounty programs. One challenge is to reduce the number of invalid (or low quality) submissions from the crowd. To address this challenge, we have built an economic model for bug bounties and analyzed multiple existing “crowd quality control” policies [8]. We also proposed a new policy and showed the advantages of this new policy over existing ones. Another challenge of running bug bounty programs is to efficiently allocate valuable but scarce hacker effort over time, and across organizations with different crowdsourcing requirements. In addition, in contrast to many crowdsourcing scenarios, bug discovery requires sophisticated participants, who are partially competing with each other. The competition often leads to multiple hackers discovering the same bug. One bug bounty platform, BugCrowd, has reported that 30% to 40% of the submissions are duplicates [2]. However, of all duplicates only the first report is rewarded. Therefore, an efficient allocation shall decrease the amount of duplicated effort, while expanding and also diversifying the manpower. We think that addressing this challenge, like other human computation problems [3], requires rigorous mathematical modeling, in order to quantify the strength and limitations of bug bounty and to design more efficient mechanisms. In this paper, we present our ongoing research on modeling and optimizing bug bounty programs.